🌻 Automated causal mapping can help answer evaluation questions
Chapter contents.

23 Aug 2025

Question for Step 3 - can automated causal mapping help answer evaluation questions?: An overview map was produced which included over 40% of the causal claims identified within the transcripts, using just 11 relatively broad factor labels.

The most central factor with the highest number of citations was Economic stress, which is a plausible result, with plausible connections to other factors.

We can use the map to identify and weigh up the evidence for contributions from and to individual factors. For example, the major contributions to Economic stress are Government policy and Covid-19, as well as “self-loops” mentioned by 46 sources, i.e. where one aspect of Economic stress was seen as causing another.

All such results depend on the (not automated) decisions made during the clustering process: how many clusters to use, whether to intervene in labelling, etc. This situation is closely parallel to decisions facing a statistician who has to identify variables for, say, structural equation modelling (Goertz 2020).

Comparison of citation frequency across timepoints was able to show that some links were mentioned significantly more than others, illustrating how this kind of map could be used to explore changes in systems (or in mental models of systems) over time.

Based on the provided factor rows, here is the analysis for rewriting the labels.

Good Minimum Set of Labels These labels cover the vast majority of the rows by grouping specific roles into broader categories.#

Other Actors to Consider These are distinct enough in the data that you might want separate labels for them, though they could be collapsed into the minimum set if necessary.#

Labels That Do Not Fit the Schema The following labels describe abstract concepts, broad contexts, or interview artifacts that do not easily accept a specific Actor or Object label without losing meaning or forcing a fit.#

Contextual/Abstract Factors: * "Passage of time" * "External crisis" (referring to COVID-19) * "Economic context" * "Societal gender norms" * "Nature of science" * "Demographic characteristics" (e.g., "being a guy vs girl") * "Cultural differences" (Abstract societal concept) Interview Artifacts (AI-related): * "AI interviewer role constraints" * "AI refusal to answer off-topic query" * "Interviewer demographic inquiry" * "Interviewer performance" * "Broad interviewer questioning"

References

Goertz (2020). Social Science Concepts and Measurement. Princeton University Press.